Papers by John D. Kelleher
Is it worth it? Budget-related evaluation metrics for model selection (L18-1)
Copied to clipboard
| Challenge: | linguistic resources can be labor-intensive, requiring great amounts of work-hours and expert annotation. |
| Approach: | They propose a machine learning model that pre-annotates or filters content before annotating it . they argue that the model with the highest F-score may not have best separation . |
| Outcome: | a case study shows that the model with the highest F-score does not yield the highest profits . the model that has the highest score does not produce the highest profit, the study shows . |